Doctoral Dissertation Rapid Unsupervised Speaker Adaptation Based on Sufficient Statistics of Hidden Markov Models

In realizing a speech recognition system robust to variation of speakers, an efficient adaptation algorithm is needed. Most adaptation techniques require many adaptation data to carry out an adaptation task. Adaptation data are often collected from the actual speaker itself in several utterances. With the time needed to gather and transcribe the adaptation utterances, together with the actual execution time of the adaptation algorithm, real-time speech recognition is difficult to realize. We propose a novel approach in solving the problem that hinders practical implementation of speaker adaptation by using only a single untranscribed utterance from the user. This unsupervised speaker adaptation approach can execute in few seconds with a significant improvement in recognition performance as compared to data-greedy and time-exhausting adaptation schemes. This thesis, details the science behind the development and implementation of the rapid unsupervised speaker adaptation based on Hidden Markov Models-Sufficient Statistics (HMMSufficient Statistics). In this approach, we process in advance the training database into HMMSufficient Statistics offline. During the actual adaptation (online), the process starts with the N-best speaker selection which is acoustically close to the user’s utterance. The HMM-Sufficient Statistics of the N-best speakers are selected ∗Doctoral Dissertation, Department of Information Processing, Graduate School of Information Science, Nara Institute of Science and Technology, NAIST-IS-DD0361218, October 1, 2006. i as adaptation data. In view of the fact that HMM-Sufficient Statistics are precomputed offline, considerable amount of computation time needed for processing is saved and re-allocated efficiently to using good-performance but computationally expensive adaptation platforms. The end result, a rapid adaptation system with good recognition performance. Experiments using Vocal Tract Length Normalization (VTLN), Maximum A Posteriori (MAP) and Maximum Likelihood Linear Regression (MLLR) were performed. Moreover we tested for robustness under noisy environment conditions such as office, car, crowd and booth noise in several signal-to-noise ratios (SNRs). In this thesis we succesfully designed a rapid unsupervised speaker adaptation that requires only a single arbitrary utterance without transcriptions and execute in 7sec of adaptation time. The proposed method is suitable for speech recognition applications where adaptation data is scarce and execution time is critical. Furthermore, we have fully integrated the proposed approach in a real application using a dialogue system where the adaptation technique is integrated and interacts freely with the recognizer and several processes in the system in a real environment condition.